Search Results: "kibi"

2 October 2010

Cyril Brulebois: Buildd fun

Being a buildd admin means one gets to sign successful build logs so that the built packages get uploaded from buildds to the archive, but that also includes filing bugs for packages which fail to build from source (Build-Attempted). Once that done, they can be marked as Failed, usually with a reason. A few words plus a bug number is usually convenient, so that people (maintainers, possible NMUers, other buildd admins) browsing https://buildd.debian.org/status/ can just click the bug number to reach the BTS. There were a few packages in Build-Attempted for alpha, for which I reported some bugs yesterday, which explains the crossing lines on the following graph: Buildd stats for alpha (That s an edited version of the buildd stats for alpha.) One has to be careful while filing bugs, since the severity depends on the target architecture(s) for which the package is failing, but also on its already being built there. Mostly, that boils down to serious for regressions (no longer builds) on release architectures, important otherwise. The only exception I m applying is packages failing on all architectures due to obviously missing build dependencies, since they can t be built again, even on the single architecture they re available for (and we want to guarantee that doesn t happen, we want to be able to binNMU them). There are currently no packages in Build-Attempted for alpha, kfreebsd-amd64, kfreebsd-i386, and sparc (architectures I take care of), but I m going to have a look at other architectures as well, since having portable software and reported (FTBFS) bugs sounds like a worthwhile goal. Flattr this!

Cyril Brulebois: October s X update

xserver-xorg-video-intel There s a previous episode regarding this driver, but let s try and get a short summary regarding UMS (User Mode Setting, old-fashioned) vs. KMS (Kernel Mode Setting, new but also Linux-specific): upstream moved from UMS+KMS to KMS only (that means a single code base and allegedly fewer bugs). UMS vs. KMS timeline:
2.9 2.10 2.11 2.12 2.12+legacy 2.12+shadow 2.13
UMS+KMS KMS KMS KMS KMS+UMS KMS KMS
In order for UMS introduced back into 2.12+legacy to be useful for the ever-buggy i8xx chips, those were blacklisted for KMS in the i915 module as of linux-2.6 2.6.32-21. The plan didn t quite work out, so we moved to the shadow version, relying on KMS again. At that point, we asked for the removal of the kfreebsd-* binaries, since they were unusable. Kernel timeline:
k 2.6.32-20 2.6.32-21 k 2.6.32-23 2.6.32-24 k
KMS KMS + blacklist for i8xx KMS
Quick summary: libdrm Due to nouveau s ABI changes, libdrm s compatibility (for nouveau users) is a funny subject. Nouveau users are expected to have the following in mind (README.Debian is there already in the xserver-xorg-video-nouveau package).
2.4.21-1~squeeze3 (sid) 2.4.22-1 (experimental)
nouveau 0.15 nouveau 0.16 aka 1.x
2.6.32 k 2.6.33 2.6.34 k
We carry a big ABI-revert patch in our 2.4.21 package so that we get the bugfixes of that version while still using the 0.15 interface, which is what we have in the sid kernel for now. If we need more stuff from 2.4.22 for squeeze, we should be able to cherry-pick them, or even adjust the ABI-revert patch quite easily. For now (and in experimental anyway), the 2.13 version of -intel is going to depend on libdrm 2.4.22, but cherry-picking fixes should be easy; making it build against sid s 2.4.21 shouldn t be hard either. xserver-xorg-video-openchrome It s now possible to start X with the openchrome driver again, thanks to Nethanel s feedback on #597379, but that doesn t mean it s usable for real unfortunately, since users still report frequent crashes. Flattr this!

30 September 2010

Evgeni Golov: What to flattr?

The month is coming to an end, and I d like to recommend you some things on flattr.com :)OpenRheinRuhr Ein Pott voll Software Die OpenRheinRuhr ist eine Messe mit Kongress rund um das Thema Freie Software . Die OpenRheinRuhr 2010 findet am 13. und 14. November im Rheinischen Industriemuseum in Oberhausen statt. Freie Software ist aus Firmen, Verwaltungen, Bildungseinrichtungen und Privathaushalten nicht mehr wegzudenken. In der bev lkerungsreichsten Region Europas bietet der OpenRheinRuhr e.V. durch Ausstellungen, Vortr ge und Workshops Informationen ber Freie Software. Auch Themen jenseits der Technik, wie B rgerrechte im Netz oder die Handhabung von Lizenzen werden behandelt. Vortr ge in mehreren Tracks sprechen Menschen mit unterschiedlichem Kenntnisstand an vom Anf nger bis zum Profi. Entwickler & Projekte bekommen die Gelegenheit zum Erfahrungsaustausch. Weitere Informationen und Anmeldem glichkeiten unter: http://openrheinruhr.deBitlBee The IRC geek s solution to instant messaging. Tunnels instant messaging traffic (supporting all popular IM protocols and Twitter) to a virtual IRC channel and virtual IRC queries. Can be installed locally, but also available as a public service for people who can t/don t want to install it.
KiBi s blog This blog mainly features Debian-related posts on various topics: X.Org package maintenance (including calls for help, and status updates towards users), Debian GNU/kFreeBSD (GNU userland running on a FreeBSD kernel), and Debian s Graphical Installer (now based on X.Org). Also some upstream stuff, like the bugzilla replica type for SD (Simple Defects).
phpMyAdmin phpMyAdmin is a tool intended to handle the administration of MySQL over the Web. It can create, rename, and drop databases, create/drop/alter tables, delete/edit/add fields, execute any SQL statement, manage keys on fields, create dumps of tables and databases, export/import CSV data, and administrate one single database and multiple MySQL servers.
Debian Backports Over the last years I did numerous backports for Debian and also run backports.org. Since September 2010 we moved backports.org to backports.debian.org so its an officialDescriptions shamelessly stolen from flattr.Some words on these things:

9 September 2010

Cyril Brulebois: SD: Bugzilla gets history support

(Disclaimer: Possibly one of my longest post ever, you may want to scroll to the bottom and only look at the pictures.) So, what s new in SD since last week? Fetching everything From http://bugs.freedesktop.org/, it was possible to download all the bugs I reported there (that means 2 ), but trying to download those reported by Julien led to a crash without any trace. Dichotomy FTW, I came up with a bunch of bugs which would lead to the same result, which confirmed that the amount of bugs processed at once wasn t the issue. Playing around with strace, it appeared that exchanges with the remote server were apparently fine, so an issue on the client-side SOAP machinery got suspected. So let s enable tracing:
use SOAP::Lite +trace => 'all';
Tada! The issue was indeed local: malformed XML got received, leading to a die call issued from the XML-RPC layer. Thankfully, one can set up a fault handler, which I used to display the range of bug IDs which triggered that bug, so that people can look into it and determine what to add to the blacklist until the underlying issue s been investigated (presumably, bugzilla s at fault). Speeding things up With that blacklisting of buggy bugs, one can then try to move to other queries, dealing with more bugs. Some examples follow:
reporter=kibi@d.o
product=xorg&component=Driver/VMWare
product=xorg&component=Driver/nouveau
product=xorg&component=Server/general
product=xorg&component=Driver/intel
Moving from 2 bugs to 10-100 bugs (Julien s or VMWare s) was OK. But then moving to several hundreds of bugs (Driver/nouveau = 500+ bugs) led to noticeable performance issues. Not to mention what happened when one reaches several thousands of bugs (Driver/intel = 2500+ bugs). Indeed, even with network exchanges cached into a local file, processing data was taking up to several dozens of minutes. I knew about Perl s -d:DProf, which helps figuring out where time is spent, but was pointed to -d:NYTProf (and its accompanying tool, nytprofhtml). Some hotspots got noticed: Things got better, but not good enough. There are several Prophet (the engine under the hood) backends, so one can play with:
PROPHET_REPLICA_TYPE=sqlite   # the default for SD
PROPHET_REPLICA_TYPE=prophet
Switching to prophet was a big win, but still not good enough. Indeed, many tiny files are written, and most of the time is spent in I/O. Although I m nothing like a performance guru, I guessed that running on an average laptop, with ext3 and its default commit interval of 5 seconds might not be helping, so I gave a quick try to -o remount,commit=60, and that seemed to help. Even though there are probably other tricks to find in that area (which hopefully won t require root privileges ), there s already a patch which landed in prophet s master branch, replacing File::Spec->catfile with an optimized version: that function alone was eating 10% of runtime In the sqlite case, disabling the auto-commit feature helped reducing the I/O load, but a proper patch is still lacking for now (running into locked database issues, or into missing tables after having created them isn t fun, so I postponed debugging that). Since performance issues looked like they could be solved eventually, I switched back to implementing missing features. Handling more than comments Currently 3 types of stuff are currently fetched from the bugzilla server: (Yes, that means that attachments are totally ignored for now.) Until now, only bug comments were considered. The first comment was used to determine a pseudo-title (using its first line), the reporter, and the creation date. This approach was chosen to try and get a basic sync working quickly, so as to get: Now, the algorithm is the following: from the bug status, determine a set of properties of interest; then walk the history backwards, and update the properties incrementally until the (presumed) initial state is reached. Then create the ticket using this initial state . Adding the incremental property changes to that initial ticket makes it possible to represent the bug s life as a list of Prophet::ChangeSet objects. That s where the fun begins, since properties in the bug status may not match properties in the bug history, so one needs to establish property correspondence. Also, some properties can be multivalued for added fun. I believe that s where most of the time is going to be spent while developing a new replica type in SD: once one knows how to get a hand on needed info on the remote server, the main question is what to do with it. For now, I decided to ignore many fields to make it possible to do a big sync like Server/general, property support will be improved later on. Screenshots Instead of pasting lengthy terminal excerpt, let s use some screenshots instead (sorry, I m not sure how to present such things in an accessible way, suggestions welcome). Cloning Julien s bugs, listing all open tickets, listing all tickets, searching using a regular expression: Cloning, listing, searching Displaying bug 42 (that s the local ID): Displaying Now, let s start the embedded web server through sd server --port 1234 and point the browser there. List of RESOLVED bugs: List of RESOLVED bugs Status and comments for bug 42: Status and comments for bug 42 History for bug 42: History for bug 42 Compared to the original bugzilla page: Same bug on FreeDesktop.org Next time Some items which need work: Flattr this!

1 September 2010

Cyril Brulebois: SD: Travel with your bugs

(For Those Who Care About An Introduction: Christine Spang gave a talk during DebConf10 about Simple Defects (SD), and blogged about it later on.) Folks maintaining Debian packages are already able to partially-clone bugs.debian.org s bug database thanks to the local-debbugs tool. But what about upstream s bug tracker? Taking a (shamelessly self-centered) example: X.Org packages are hosted on FreeDesktop.org s bugzilla. Thanks to SD, it s possible to fetch bugs from there as well! Here s the obligatory picture: SD example This means that you can browse/search them locally while being offline (or well-connected, but without having to use that !$\ %$^ bugzilla web interface). Many of the replica types support both reading and writing, meaning you can also queue some changes locally, and push them later. Currently, sd help sync says that read-write support is available for RT, Hiveminder, Trac, Google Code, and GitHub. There s also read-only support for redmine. Debbugs is being worked on, see Christine s blog post about her SD talk for more info. Given there was no support for bugzilla, I had a quick look and reported my findings. The main point being: \o/ Bugzilla s XMLRPC \o/ A little while later (I m not exactly fluent in Perl ), I came up with a tentatively-mergeable branch adding preliminary read-only support for bugzilla. There s still a lot of work, but I m trying to work on it on a regular basis, adding support for more properties, and fixing bugs (tests should be written some day, too). Flattr this!

10 August 2010

Asheesh Laroia: "Debian for Shy People": What's next

What do you do when you have a technical question that you're embarrassed to ask? The first Sunday of Debconf, I led a birds of a feather (BoF) session called Debian for Shy People. The conference team scheduled it on "Debian Day," a pre-conference day that was open to the public and still had plenty of Debian Developers in attendance. I just uploaded the slides to the "Penta" page for the talk. I led it because of my own experience. In 2004 or so, I saw Debian as the cool kids' club, that awesome project that I wished I could be a part of. By 2006, I managed to get over myself, read the New Maintainer's Guide, and find a way to get involved. As of mid 2009, I am a full-blown Debian Developer. I have real ultimate power. But I sometimes do still feel hesitation akin to "Imposter Syndrome". (A bunch of people at Debconf didn't really believe I'm "shy," since I asked a lot of questions at the conference. At core, I don't naturally believe that the things I say are worth hearing, but I patch over this hesitation. Sometimes I speak too much, and then I feel ashamed of burdening everyone. But anyway, this is about Debconf not, me -- so moving on....) In the past year of being a Developer, one thing I've seen is that other contributors ask me privately for help. Rather than blast the public lists like debian-mentors, they email or IRC private-message me, or SMS me, or find me at a Linux Users Group event. I'm lucky to know these people, and they're lucky to have me as a safe person to ask questions of. Moreover, Debian is better because these people could move past their confusion to make a technical contribution. I began the BoF session by talking about when someone asked me for help. Then I asked, "How many of you have someone you can ask embarrassing questions of?" Of the forty people crammed into Schapiro 414, two people' raised their hands. One person put it plainly, "I don't know anyone else who does Debian." It reminded me of a fact that Karen discovered when she was doing market research for us at OpenHatch: the vast majority of free software programmers know zero other people who do free software. I had seen the figure; we even used it in a talk to try and convince venture capitalists to fund OpenHatch last year. But I didn't really feel it until I heard it from a room full of Debian contributors. I structured the BoF in two parts: First, I talked in front of some slides to set the tone properly, and then we enjoyed open discussion. As I was preparing thoes slides, Daniel Morais asked me, "What's the point of having the session? Why not just come up with some ideas, implement them, and not bother also talking about it at the conference?" I had considered this; I decided I wasn't self-confident enough to start implementing ideas without talking to people to make sure I wasn't the only one who saw a problem. But I discovered another benefit of giving the talk: people who want to make Debian more welcoming knew to reach out to me. So here are some thoughts that came from our discussion (and later discussions during the conference): I set up an Etherpad document on cjb's OpenEtherpad.org. This is what we learned together: One idea I had before the BoF was to create a discussion area that was safe for all questions, even if they seem silly. We talked for a while about what name that would take, if it were to become a new IRC channel. We reached something of a conclusion, but in the conference that followed Emmet Hickory offered to help make the debian-mentors IRC channel friendlyer. I think that's the best direction to take things, so the next step is for him and me to write up what we want and send a note to the debian-mentors email list explaining our vision. In the Etherpad document, people discussed the idea of doing Debian discussion over XMPP (also known as Google Talk, also known as Jabber). We weren't sure how such a place would get critical mass; someone briefly mentioned the idea of an IRC/XMPP gateway. I actually think this discussion is along a very reeasonable path, namely discovering what discussion method(s) Debian contributors want to use. (That might explain why I'm now an admin on forums.debian.net.) We also briefly discussed the idea of an anonymous question-answering service. I realize now that I'm not going to be able to have time to run that, but I still think it'd be a really cool idea. Biella would remind me that Debian is already successful at bringing in new contributors. I agree! As a free software project, we have an enormous number of participants. This is a really good thing, and we're clearly doing something right. The purpose of this talk was to figure out how to make contributing to Debian less stressful for those who participate. Truly, a "Debian for Shy People" effort isn't about shy people. It's about the moments of self-doubt we all have in which we don't know what to do and are too embarrassed to ask. I think that if the project more friendly, we can find more participants, make better use of our current ones, and see improvements to our diversity. Whew, that was long. What do you think of all this?

18 July 2010

Cyril Brulebois: Mraw?

That s a recurring question I get asked: What does mraw mean? Over 2 years ago, I changed my mail signature from the traditional:
--
$FULLNAME
to:
Mraw,
KiBi.
I became a DD back then, so I put my now-become-my-login-as-well nickname there. About the mraw part, I kind of like felines in general, and felis catus in particular. Onomatop ias like maw or meow are nice, but that s even cuter when there s some purring happening at the same time, hence the r addition in there.

4 July 2010

Cyril Brulebois: We need *you*! (redux)

We need you! (redux) The X Strike Force needs you! (redux) This time, the focus is on the xserver-xorg-video-intel driver. KMS and UMS are going to be used heavily below, one may want to check the mode-setting wikipedia article and the modesetting page on X.org. Intel follows a 3-month cycle for its video driver releases. Some comments on the last ones: It d be nice to have users install 2.12 from experimental and see how it goes from here. If regressions are found, we should be able to try hard(er) to get them fixed, so that we can move 2.12 to unstable and release with it. If there are too many of them, and if they re not going to be fixed on time, we might stick with 2.9. Pulling the driver from experimental should be quite easy: libdrm* packages are going to be upgraded, but that s mostly it (meaning the server itself will stay the same, and rolling back should be as easy as downgrading the driver and the libdrm* packages). A note on KMS and UMS: a benefit of sticking with 2.9 was being able to switch back to UMS in case KMS wasn t usable for some cards. Unfortunately, popular/well-spread cards like i8xx aren t well supported yet, that s why Chris Wilson (upstream) is considering introducing UMS support again for that kind of blacklisted cards. This way, it should be possible to move to 2.12 (including Chris work) without leaving some users behind. Feedback appreciated as follows: Thanks already!

18 May 2010

Bernd Zeimetz: gimp-plugin-registry package updated

Yesterday I finally found the time to update gimp-plugin-registry, the (hopefully!) largest and best collection of plugins for The GIMP. As usual here comes a short summary of the changes: Hope you like my work, suggestions and bug reports are welcome as usual!

1 May 2010

Sean Finney: Nouveau, Gallium3D, and Compiz on Debian

update: 20100503: It's been brought to my attention that there might have been a regression in one of the libraries, mesa, maybe, so that master might not be as reliable a reference point for this guide. so, just in case, here are the commit hashes that were used from the different projects: for the non-git-savvy: git checkout -b mybranch <hash> we now return to your regular broadcasting... A week or two ago I figured it was about time that I took the plung and tried out the latest nouveau drivers from experimental on my laptop. Unfortunately, it meant bidding adieu to compiz (apart from the bling it does actually provide some pretty damned useful features). I thought I'd give it a shot at least for a while on principal, since I've always had to think of the nvidia drivers as a necessary evil, and their lack of proper xrandr support is really, really annoying. Since then I haven't noticed any problems, apart from the following message after every suspend/resume:
[  931.213239] Uhhuh. NMI received for unknown reason 80 on CPU 0.
[  931.213240] You have some hardware problem, likely on the PCI bus.
[  931.213242] Dazed and confused, but trying to continue

No idea what's causing it, but otherwise the system is stable and functional. If anyone knows what can be done about this, I'd appreciate hearing from you. My graphics card, by the way:
01:00.0 VGA compatible controller: nVidia Corporation G72M [Quadro NVS 110M/GeForce Go 7300] (rev a1)

In the time since then, the nouveau driver has made its way into unstable, and a lot of distros (ubuntu, fedora, ...) are even using it as the default driver. So before going any further with the 3d/compiz stuff, I highly recommend that you get the packaged version working first. This will not only make sure that the basic driver works, but it will also make recovery/rollback a lot easier. Since it's all packaged in the standard repositories, it's not much harder than installing a few packages and editing a few files... so come on give it a try :) Getting the 2d driver working in Debian sid Really, this is so easy it's hardly worth writing up, but i'm running with the -v flag this morning... Make sure you don't have any reference to nvidia in /etc/modules or other places that might cause it to be automatically loaded. Then remove the proprietary glx bindings, which would otherwise interfere with the nouveau driver.
sudo apt-get remove nvidia-glx

Next, you'll need to install a kernel that includes updated drm support. Normally this would require a kernel >= 2.6.33.1, but the debian kernel maintainers have graciously backported this code to the 2.6.32-4 kernel package. note this isn't the default kernel version in testing/unstable at the time of writing so you have to explicitly install it along with the nouveau driver.
sudo apt-get install linux-image-2.6.32-4-amd64 xserver-xorg-video-nouveau

If you are already running this kernel and using the proprietary nvidia drivers, via module-assistant or otherwise, you /might/ need to uninstall them first. On my system I left them installed in the old kernel version but did not install them on the new kernel version. Required xorg configuration I don't believe that nouveau is picked by default if available at the time of writing, so you have to have the following in your xorg.conf:
sudo sh -c "cat > /etc/X11/xorg.conf" << EOF
Section "Device"
    Identifier     "nVidia Corporation G72M [Quadro NVS 110M/GeForce Go 7300]"
    Driver         "nouveau"
EndSection
EOF

This was all I needed to do, apart from rebooting into the new kernel. For those who don't want to venture outside the packaging system, this is about as far as you can go for the time being. Though really, if you're bothering to read this far along, you're probably more interested in what comes next :) Getting 3d working (without voiding your distro warranty) So I thought I'd give the 3d-accelerated gallium a go. Their wiki page certainly makes one think twice, but at this point I was seriously fiending to get my bling back (well, the compiz scale plugin, anyway) and figured it wouldn't hurt too much to try. Via google and trawling around in various forums I found a number of how-to documents for getting Gallium3d on debian/ubuntu systems, but most of them recommended doing things in a way that wasn't easily reversible and that could totally hose your system in a not-so-hypothetical worst-case scenario. It's pretty hard to roll back a make install done into /usr, for example, and subsequent package upgrades can leave your system even worse off as they will probably clobber half of what you just did, leaving the other half totally broken. And if you have a laptop which requires the gui for networking (i.e. network-manager), then fixing/rolling-back the changes is not entirely trivial, since you'd likely have to re-download the various drm/mesa/xorg packages. So enter the tried and true unix utility stow (apt-get install stow, of course). stow is a utility that lets you install software into arbitrary folders, and then manage the system-wide installation of the software via a farm of symlinks pointing at the installed location. It's probably more well known among the sysadmin crowd than the desktop user crowd, but it's incredibly useful for the task at hand as it will let us easily install multiple copies of different software in a way that's very easy to update and/or roll-back. RTFM stow(1) for more information. So, on to the juicy stuff... Install needed software, remove problematic packages, setup, etc At least on my system, I had to remove libdrm-dev to keep things from accidentally linking against it instead of the updated version that we install later on below. There's probably a way to override this with CFLAGS but I figure this is simpler.
sudo apt-get remove libdrm-dev
sudo apt-get install stow build-essential xorg-dev git-core libtool mesa-common-dev automake autoconf 
mkdir ~/nouveau

Compile a linux-2.6.34 RC kernel
cd ~/nouveau
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
cd linux-2.6
git remote add -f nouveau git://anongit.freedesktop.org/nouveau/linux-2.6
git checkout -b nouveau-master nouveau/master
make menuconfig

Enable device drivers -> staging drivers -> nouveau. Make any other changes you might want to make (I just took defaults for everything else).
make deb-pkg

The linux-firmware-free debian package ships some of the same stuff from the generated linux-firmware-image package. I decided to opt for the upstream firmware, though I'm not sure that this is necessary:
sudo apt-get remove linux-firmware-free
sudo dpkg -i ~/nouveau/linux-image*.deb ~/nouveau/linux-firmware-image*.deb
sudo update-grub

Install a copy of dri2proto
cd ~/nouveau
git clone git://anongit.freedesktop.org/xorg/proto/dri2proto
cd dri2proto
./autogen.sh --prefix=/usr/local/stow/dri2proto-20100428
sudo mkdir -p /usr/local/stow/dri2proto-20100428
sudo make install
sudo stow -d /usr/local/stow dri2proto-20100428

Install a cutting-edge version of libdrm
cd ~/nouveau
git clone git://anongit.freedesktop.org/git/mesa/drm/
cd drm
./autogen.sh --prefix=/usr/local/stow/drm-20100428 --disable-intel --disable-radeon --enable-nouveau-experimental-api
make
sudo mkdir -p /usr/local/stow/drm-20100428
sudo make install
sudo stow  -d /usr/local/stow  drm-20100428

Likewise for mesa
cd ~/nouveau
git clone git://anongit.freedesktop.org/git/mesa/mesa
cd mesa
./autogen.sh --prefix=/usr/local/stow/mesa-20100428  --enable-debug --enable-glx-tls --disable-asm --with-dri-drivers= --enable-gallium-nouveau --disable-gallium-intel --disable-gallium-radeon --disable-gallium-svga --with-state-trackers=glx,dri --with-demos=xdemos,demos,trivial,tests
make
sudo mkdir -p /usr/local/stow/mesa-20100428 
sudo make install
sudo stow -d /usr/local/stow mesa-20100428

And now the xorg driver...
cd ~/nouveau
git clone git://anongit.freedesktop.org/git/nouveau/xf86-video-nouveau/
cd xf86-video-nouveau
./autogen.sh --prefix=/usr/local/stow/xf86-video-nouveau-20100428
make
sudo mkdir -p /usr/local/stow/xf86-video-nouveau-20100428
sudo make install
sudo stow -d /usr/local/stow xf86-video-nouveau-20100428

Configuration Configure xorg to look in alternate location for xorg modules and set up DRI
sudo sh -c "cat > /etc/X11/xorg.conf" << EOF
Section "Files"
    ModulePath "/usr/local/lib/xorg/modules,/usr/lib/xorg/modules"
EndSection
Section "ServerFlags"
    Option "GlxVisuals" "all"
Endsection
Section "Device"
    Identifier     "nVidia Corporation G72M [Quadro NVS 110M/GeForce Go 7300]"
    Driver         "nouveau"
EndSection
EOF

I've /heard/ that disabling TV out is a necessity on some cards, though I haven't seen this myself:
sudo sh -c "cat > /etc/modprobe.d/nouveau.conf" << EOF
# not entirely sure this is necessary, but heard it helped with vsync
options nouveau tv_disable=1
EOF

Finally, the one thing that can not be done in /usr/local is making the nouveau DRI driver available. From what I could tell the path that mesa uses to find the DRI libraries is hard-coded (corrections welcome). Since you won't already have this file (otherwise, why would you be following along here?), just drop in a symlink to the file in /usr/local:
sudo ln -s /usr/local/lib/dri/nouveau_dri.so /usr/lib/dri/

Finally, reboot, and hope for the best :) Riding the bleeding edge: how to upgrade later on Again, stow really shows its usefullness here. Using drm as an example case, upgrading an individual component would look something like:
cd ~/nouveau/drm
git clean -dfx
git pull
today= date +%Y%m%d 
./autogen.sh --prefix=/usr/local/stow/drm-$today --disable-intel --disable-radeon --enable-nouveau-experimental-api
make
sudo mkdir -p /usr/local/stow/drm-$today
sudo make install
sudo stow -d /usr/local/stow -D drm-20100428
sudo stow -d /usr/local/stow  drm-$today

Note how the old version is still installed, so you can switch components around pretty freely. Abort! Abort!: How to roll back changes In case your system becomes totally unusable, or you've decided that you just want to go back to the safe and cozy world of what's provided by the debian packages, simply do the following: Assuming that you're using the nouveau driver from unstable, this should be all that is necessary. Conclusion Overall impression I am very, very happy with the resulting setup. I was led to believe that this whole thing was much more unstable and unusable than it actually is! While there are certainly some glitches and missing features, I feel comfortable enough using it that I don't have plans to switch back. But if I find myself in a situation where I can't use the new setup, rolling back shouldn't be a problem either. I figure I'm happy with what I have now and it will only get better. What works, at least to a "kinda works" level. What still doesn't work Further reading, references Thanks A couple shoutouts to those who helped me with this on the way: Feedback / Corrections Unforunately, I've had to disable comments for the time being due to overwhelming quantities of spam. So please drop me a line via email or on IRC (seanius on oftc/freenode).

15 April 2010

Bernd Zeimetz: gimp-plugin-registry package updated

Yesterday I finally found the time to update gimp-plugin-registry, the (hopefully!) largest and best collection of plugins for The GIMP. As usual here comes a short summary of the changes: Hope you like my work, suggestions and bug reports are welcome as usual!

30 March 2010

Cyril Brulebois: We need *you*!

We need you! (Thanks to Mauro for his nice thick-swirl design, fitting better than the traditional-swirl one.) The X Strike Force needs you! Thanks to the kernel team s work on getting DRM stuff backported to 2.6.32 kernels, and thanks to Sven s work on preparing both libdrm and xserver-xorg-video-nouveau, binaries are now available for users to try out! One should note I m a bit uncomfortable with publishing those binaries since I wasn t able to test those packages myself (I haven t got such hardware at the moment), but Sven reported it was working fine for him, and I received another successful report from a daring user (thanks, Duck!). Given the current ftp-master outage, I ve built i386 and amd64 binaries (using source from there), which are available through my people.debian.org page. libdrm* packages can be found in experimental, and 2.6.32-4-$arch kernels (built from linux-2.6 revision 2.6.32-10) in sid. Please keep in mind those are meant to be experimental packages. Depending on your hardware, you might have to download and install some firmware. Details can be read in this message, in that one, and in that other one (which arrived when I started writing this blog post, nice; that also means you may want to keep an eye on that thread). I guess versioning might be reworked at some point, you might have to manually select experimental or unstable packages once they hit the archive, but early testers probably are going to find their way, right? Feedback appreciated through debian-x@. Thanks already!

12 March 2010

Cyril Brulebois: Ponies!

As a follow-up to Tolimar s post, here are some ponies for ftpmasters: Pony #1 Pony #2 (Clicking on an image leads to the relevant flickr page.) Those crazy folks are almost ACCEPTing X11 libraries with added udebs faster than I m uploading them. Source packages needing an upload include cairo, pango1.0, gtk+2.0, udev, and xorg-server. Then we should be able to deal with udebs maintained by the d-i team. Anyway, back to the initial topic: Ponies, thanks & kudos to ftpmasters!

7 March 2010

Brice Goglin: Debian/X.org notes - Bug triaging while waiting for DRM 2.6.33

Almost nothing interesting happened recently in X.org in Debian. But interesting things are coming soon.
First, radeon KMS and DRI2 will enter unstable soon. xserver-xorg-video-radeon 1:6.12.191-1 is currently in experimental. People seem to be happy with it so far, and upstream is taking very good care of bug reports as usual. The next 2.6.32 kernel will contain DRM from 2.6.33. It first means that the radeon KMS driver not in staging anymore. Once this new kernel is uploaded, I'll put the new xserver-xorg-video-radeon in unstable (6.13.0 is expected soon, but 6.12.191 already looks good so far). DRM from 2.6.33 will also brings nouveau support. It means that we will build libdrm-nouveau and upload a new xserver-xorg-video-nouveau. However, it also means that we need somebody to maintain this. And nobody in the team has a nvidia board to test packages so... If you want nouveau in Debian, please help.
While waiting for all these, we have been triaging the BTS a bit. Kibi is helping a lot by triaging recent intel bugs (many regressions fixed in recent kernels). I spent some time during the week-end triaging some old bugs. I closed more than a hundred of them, and pinged another hundred. We still have more than 1100 bugs open. It is not so bad compared to 1500-2000 when nobody maintains X (aka often), but still way too much. Some of my bug closing might look a bit rude. But we had so many bug reports a couple years ago that are irrelevant today. Keeping them open would be meaningless. For instance, many input problems are obsolete since a lot of the input code was rewritten, we switched to input-hotplug, and then hal to udev. Another example is intel lockups (we had a lot of them after driver 2.2 arrived). But XAA and EXA were dropped in favor of UXA, DRI1 was dropped for DRI2, and KMS arrived. So it's useless to keep these obsolete and irrelevant bugs that cannot be debugged nowadays.
As usual, the Debian X team needs a lot of help. Again, if you want nouveau in Debian, please help.
(Permanent link)

28 February 2010

Cyril Brulebois: Where have you been

Time for a little end-of-the-month looking back. (Looking back o-ver my shoul-der ) What about X11-based graphical installer? After the one-week rush early this month, a summary of my findings was posted on debian-boot@, including udebs, patches, and source packages, labelled as v1. People seemed to like the idea, and I received a bunch of comments to fix or improve various things. It took some time to implement everything in a satisfying (at least to me) manner, but I reached a point where I was confident enough to publish a v2 set of udebs, patches, and source packages. That was also announced on debian-boot@, and then re-announced once I ve been able to rebuild the image with speakup enabled again. Hopefully, that ll be (more or less) the final plan, and d-i folks should either give us a green or red light based on that v2 set. Some details will have to be ironed out (some items were mentioned in both v1 and v2 summaries), but they shouldn t prevent us from starting adding the needed udebs; or at least not too long. Speaking of X11, xserver-xorg-video-intel is buggy as hell! Basically, yes, and that s known. Many bugs were due to kernel issues, and regressions got fixed in the series of 2.6.32.y stable updates. Usually, upgrading to at least 2.6.32-3-$arch (from linux-2.6 s revision 2.6.32-9) helps, as well as running latest libdrm, xserver-xorg-core, and mesa from sid. Trying with some 2.6.33-ish kernels from experimental would be nice as well. You re of course very welcome to keep us posted through the bugs you opened already, closing them if appropriate. And before reporting more bugs, please make sure you re running recent versions of all those components. That should help people walking the list of remaining bugs to deal with them more efficiently. A very annoying bug which can still be experienced with an up-to-date sid environment is described in freedesktop s #26723.

15 February 2010

Gustavo Noronha Silva: WebKitGTK+, and the Page Cache

So, one of the things I get to do during work hours for Collabora is to contribute code, and do maintenance tasks for WebKitGTK+, and have been doing so since early last year, working on all kinds of things, from improving the network backend to handle the real-world web, to fixing scrolling problems, while reviewing patches from the many awesome developers who have been joining us (more on that later =D). One of the big features I have worked on this past month or so, along with Xan Lopez is the Page Cache. The page cache is a feature of web browsers that makes going back, and forward between pages in the same view very fast. It s better explained in this post, but to summarize, the idea is that instead of destroying all the work you have done since downloading the resources, and having to reparse/rebuild the structures the view uses to display the page from the cached resources, you hit pause on the page, and store the whole thing as is, and when coming back to it, you just hit play. You can see in the video two instances of Epiphany, one with the page cache enabled, one with it disabled. Easy to see which was has it enabled. Thanks to KiBi for the suggestion regarding a page that shows this easily =D. <video controls="controls" height="302" src="http://kov.eti.br/media/webkit/page-cache.ogv" width="450"></video> We initially thought we had this feature enabled, since our initialization functions (that exists since before the current maintainers were involved) did setup the number of desired pages in the cache, but during the hackfest we held in December we found out we were fooled all this time. Enabling the page cache does make going back faster, but also made lots of things become unstable and crash. Since then, we have been working on figuring out all the problems, and fixing them, using help from adventurous users of in-development software ;D. I believe we re now at a point in which I can happily declare the GTK+ port has a working page cache in trunk! If you re interested in the nasty details, bear with me! Let me go back in time a bit, and show you what problems we had. First, some background: the GTK+ port deviates a lot from the other ports when it comes to scrolling. This is because, when designing this part of the port, Holger Freyther had a very nice idea in mind: that the WebView should be a first-class citizen GTK+ scrollable widget. Meaning it would use GTK+ s adjustments for scrolling, and be able to interact with any parent scrolling widget, be it a GtkScrolledWindow, or a MokoFingerScroll. We cannot just throw away all the rest of the scrolling code in WebCore, though, that deals with all the details related to interacting with the DOM, and JavaScript code. This means our WebView contains adjustments that need to be set, and unset on our port s version of WebKit s own representation of the view, called the FrameView, to interact with it, and to get updates on the bounds of content, and such. For every load, in the non-page-cache case, a new FrameView is created, the previous one is destroyed - this means we need to set the adjustments on every load. The problem starts when you have the page cache enabled, because the code path used to do what is called commit the load of a cached page (that is, start replacing the content that is currently being displayed by the one that should now be displayed) is completely different, and we were not setting the adjustments on this new view, so we started with that. But all was not well. We were still having weird behaviour with scrollbars disappearing, and becoming the wrong size, and worse, crashes when back was hit. We then started investigating in more detail how it is that the page cache does its magic, to try and figure out the source of all evil. It turns out that when you leave a page that can be cached, the existing FrameView is no longer destroyed - it is stored as is in a CachedFrame to be restored if you go back, and a new one is created for the new page. This was having the undesired effect of having the adjustment be set in more than one FrameView at once, causing all kinds of (predictable, after we knew for real what was going on) unwanted effects. Thus, we reworked the code to make sure the adjustments are only ever set in one FrameView at once, making sure they are unset when the FrameView is being frozen, and reset when it s being restored from the page cache. Last, but not least, it was discovered that going back to a page that contained resources with data: URIs (such as Google results pages which contain a small number of image hits) also caused a crash. This was because our network backend was not storing the data: URI in the ResourceResponse objects it fed into WebCore. The page cache relies on those responses to recreate the requests it uses to artificially replay the load when restoring the page from the page cache, so we fixed that as well. What can be taken from all this? Building browsers is a lot of hard work! I can t think how we could deal with this level of complexity without the awesome testing suite of WebKit. The good news is all of those issues I talked about in this post are now covered by the automate tests that run as part of the normal buildbot cycle in our bots, so we re covered for the future, at least for these specific problems =D.

9 February 2010

Kees Cook: rng-tools with TPM

In Ubuntu, I uploaded an rng-tools that supports the RNG in TPM devices (my patch is waiting in Debian). This hardware is available on a bunch of systems, including several Thinkpads and the Intel Q35, Q45 and newer main boards. While most TPM RNGs aren t really heavy-duty hardware RNGs, they are at least a mild source of randomness. I ll be using an entropy key eventually, but for now, the TPM can supplement my collected entropy. /etc/default/rng-tools:
HRNGDEVICE=/dev/null
RNGDOPTIONS= hrng=tpm fill-watermark=90% feed-interval=1
After it s been running a bit:
Feb 8 19:10:51 linux rngd[13143]: stats: bits received from HRNG source: 6180064
Feb 8 19:10:51 linux rngd[13143]: stats: bits sent to kernel pool: 6166144
Feb 8 19:10:51 linux rngd[13143]: stats: entropy added to kernel pool: 4624608
Feb 8 19:10:51 linux rngd[13143]: stats: FIPS 140-2 successes: 309
Feb 8 19:10:51 linux rngd[13143]: stats: FIPS 140-2 failures: 0
Feb 8 19:10:51 linux rngd[13143]: stats: FIPS 140-2(2001-10-10) Monobit: 0
Feb 8 19:10:51 linux rngd[13143]: stats: FIPS 140-2(2001-10-10) Poker: 0
Feb 8 19:10:51 linux rngd[13143]: stats: FIPS 140-2(2001-10-10) Runs: 0
Feb 8 19:10:51 linux rngd[13143]: stats: FIPS 140-2(2001-10-10) Long run: 0
Feb 8 19:10:51 linux rngd[13143]: stats: FIPS 140-2(2001-10-10) Continuous run: 0
Feb 8 19:10:51 linux rngd[13143]: stats: HRNG source speed: (min=5.207; avg=6.145; max=6.200)Kibits/s
Feb 8 19:10:51 linux rngd[13143]: stats: FIPS tests speed: (min=66.925; avg=75.789; max=112.861)Mibits/s
Feb 8 19:10:51 linux rngd[13143]: stats: Lowest ready-buffers level: 0
Feb 8 19:10:51 linux rngd[13143]: stats: Entropy starvations: 308
Feb 8 19:10:51 linux rngd[13143]: stats: Time spent starving for entropy: (min=3150263; avg=3178447.994; max=3750848)us
And now the kernel entropy pool is high:
$ echo $(cat /proc/sys/kernel/random/entropy_avail)/$(cat /proc/sys/kernel/random/poolsize)
3968/4096

8 February 2010

Kartik Mistry: My Hero


* Cyril Brulebois (kibi) is my hero

6 February 2010

Cyril Brulebois: G-I, part 4

A picture is worth a thousand words: A slightly longer story follows, resuming from the previous episode, where Gtk seemed to be working. Making sure Gtk actually works Since the udebs are quite fat already, I decided to put the contents of the gtk2.0-examples package into a new revision of the new libgtk-x11-udeb package. gtk-demo was running fine (if one forgets that this demo application is really easier to deal with if one uses a window manager), although image handling was broken. Julien and I spent quite some time double-checking that the libpixbufloader-png.so plugin was present; that both previous DirectFB-based udeb and new x11-based udeb were very similar; that configuration was generated properly; we went so far as to include strace in the netboot image, and to check the configuration file was open() d, but the plugin was never to be even looked for. All this resulting in a Unrecognized file format message every time. Josselin, our Messiah, enlightened us: although Gtk was previously using some configuration files to determine which pixbufloader plugin to use, it still continues to access those files but relies on a MIME database in the end. Installing the host s /usr/share/mime/mime.cache database did the trick (we have a proper fix pending), and all of a sudden, we got image support (which is needed for the logo_installer.png displayed on top of every screen, as well as for some icons during the installation, like the final one). Tweaking d-i itself Julien took care of rootskel-gtk. It was mostly about: Another easy step was cdebconf. Mostly: search and destroy on the few bits using DirectFB. An interesting issue we had was that even though a proper cursor theme was installed and made the default, we only had a black cross as X pointer. That s where we learnt it s needed to manually set a cursor for the root window, which can be done this way:
GdkCursor *cursor = gdk_cursor_new(GDK_LEFT_PTR);
gdk_window_set_cursor(gdk_get_default_root_window(), cursor);
gdk_cursor_unref(cursor);
Tweaking gtk2-engines was trivial, just a matter of building against the x11 flavour of Gtk instead of the directfb one. Some substitutions in debian/control and debian/rules, and that was done. Julien also took care of tweaking console-setup so that keyboard-configuration s and console-setup-udeb s postinst scripts call setxkbmap when $DISPLAY is set. Meaning we were able to replace kbd-chooser with console-setup-udeb, leading to a correct setting within X right after having selected the layout! Reverting d-i s r61645 (which disabled Gtk-based d-i images) was trivial (although while looking back, it was probably unnecessary), and after having messed around with the packages lists (there s a lot of local udebs at this point), especially to add some fonts, we managed to get an image which seems to work fine (at least in my VirtualBox environment). I tried Chinese, Japanese, Arabic, and French installations, and all of them seem to work fine (installed systems were bootable after an installation in the selected language) while issues were limited to some rendering or translation glitches, which didn t seem critical at all. Next steps Since there were some iterations, and since the trial-and-error approach was chosen, the first goal is to generate clean patches against all modified packages, and make sure a single walk through all packages in a clean chroot is sufficient to generate a working d-i image again. Once that done, which should probably considered (at least, I hope!) quite a good base already, the idea is to look into the various udebs to see whether there are some parts that could be disabled (maybe introducing a trimmed-down flavour especially for the udeb in some non-flavoured-yet libraries), so as to reduce the size of the resulting image. In the meanwhile, one can fetch the current hand-built netboot image by browsing http://people.debian.org/~kibi/di-x11/. It s only available for amd64, and weights 23MB. There s a sha256sum as well as a detached GPG signature in the same directory.

5 February 2010

Cyril Brulebois: G-I, part 3

Last night was the third one spent on playing around with X in a d-i environment. Making sure X actually works So far, running X was nice already, but is it really possible to run some applications inside? Bad news is that many trivial applications (xeyes, xclock, ) pull more dependencies than previously packaged. Finally, xev was chosen, and it indeed works fine (provided one makes sure it gets the focus). X was crashing when the (only last) client was exiting, and that wasn t normal. Thankfully the bug was reproducible outside d-i and Julien sent a patch a few minutes later. In the meanwhile, using -noreset did the trick to keep X running. Another way to run X applications there, is to point the host environment to the guest environment using DISPLAY=$GUEST:0 xterm. With VirtualBox, I followed those steps: Moving forward: cairo, pango, gtk The next step was to rework some udebs. Legend: Current situation, keeping only the relationships between udebs of interest: I took care of gtk+2.0 while Julien hacked pango1.0 and cairo. Some more libraries were needed, but being X libraries, tweaking the packaging was as trivial as for the first X libraries, and it only took a couple of minutes: libxcomposite, libxdamage, and libxinerama. Once those added to localudebs/ and to pkg-lists/local, and once the image rebuilt, I prepared a tiny Hello world -style application: Pango wasn t quite happy at first (message wrapped):
(hello:1991): Pango-WARNING **: failed to choose a font, expect
              ugly output. engine-type='PangoRenderFc', script='latin'
Indeed, since no fonts were included, there was a tiny issue. No big deal, adding ttf-dejavu-udeb did the trick. I wanted to play around with a non-Latin language, so I asked for an example (thanks, Theppitak!), and once ttf-thai-tlwg-udeb added as well, one gets: Of course, there are many more features to test, but that sounds like we re going in the right direction (and this is not a joke about RTL). Next step Have a look at the green boxes, which may require to have a closer look at g-i s integration within d-i (it might be needed to see how it s supposed to be started, that might help testing the tweaked packages).

Next.

Previous.